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ABSTRACT 

Semantic Web technologies have been applied in educational 
settings for different purposes in recent years, with the type of 
application being mainly defined by the way in which knowledge is 
represented and exploited. The basic technology for knowledge 
representation in Semantic Web settings is the ontology, which 
represents a common, shareable and reusable view of a particular 
application domain. Ontologies can support different activities in 
educational settings such as organizing course contents, 
classifying learning objects or assessing learning levels. 
Consequently, ontologies can become a very useful tool from a 
pedagogical perspective. This paper focuses on two different 

experiences where Semantic Web technologies are used in 

educational settings, the difference between them lying in how 
knowledge is obtained and represented. On the one hand, the 
OeLE platform uses ontologies as a support for assessment 
processes. Such ontologies have to be designed and 

implemented in semantic languages apt to be used by OeLE. On 
the other hand, the ENSEMBLE project pursues the development 
of semantic web applications by creating specific knowledge 
representations drawn from user needs. Our paper is 

consequently going to offer an in-depth analysis of the role played 
by ontologies, showing how they can be used in different ways 
drawing a comparison between model patterns and examining the 
ways in which they can complement each other as well as their 
practical implications 

KEYWORDS: LEARNING, ASSESSMENT, EDUCATIONAL 
TECHNOLOGY, ONLINE SYSTEMS, INTERNET 

1 INTRODUCTION: ONTOLOGIES AND THE 
SEMANTIC WEB IN ELEARNING 

The Semantic Web aims to add semantic information to web 
contents in order to create an environment where software 
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agents can perfonn tasks efficiently (Berners & Hendler, 2001). 
The Semantic Web proposes the idea that web contents are 
defined and linked not only for visualization but also to be used 
by applications. That is why the Semantic Web represents a 
promising technology to implement e-leaming systems. 
Furthermore, Stojanovic, Staab, and Studer (2001) showed that 
the Semantic Web meets the basic e-leaming requirements, 
namely: speed, just-in-time and pertinent learning. The 
appropriateness of Semantic Web technologies for developing 
eLearning systems is also supported by the research work 
undertaken in the last years from different perspectives (see 
Fensel, Staab, Studer, Van, & Davies, 2003; Devedzic, 2006; 
Bittencourt, Costa, & Silva, 2009; amongst others). 

The Semantic Web has as its aim to go beyond the limits of 
the current web by introducing explicit descriptions of meaning, 
the internal structure and the overall structure of the contents and 
services available on the WWW. Before the chaotic growth in 
resources and the absence of any clear organization in the web 
today, it advocates a classification, structuring and annotation of 
resources with explicitations that can be processed by semantic 
machine. 

A number of technologies are needed for the success of the 
Semantic Web. The Semantic Web stack includes languages and 
standards ranging from how characters are represented to how 
security can be guaranteed. Flowever, ontologies have become 
the cornerstone of the technology required if all the information 
is to be made understandable by machines. A large number of 
definitions for ontology can be found in the literature (see 
Gruber, 1993 or Van Heist, Schereiber, & Wielinga, 1997, for 
instance). Ontologies define common, shareable and reusable 
views of a domain, giving meaning to information structures that 
are exchanged by information systems (Brewster & O'Flara, 
2007). Ontology can be seen as a semantic model that contains 
concepts, their properties, interconceptual relationships, and 
axioms related to the aforesaid elements. In practical settings, 
ontologies have become widely utilized because they are 
reusable and shareable (see Fernandez-Breis & Martinez-Bejar, 
2002 and Brewster & O'Hara, 2007, amongst others). 

On the e-Learning side, some standards and recommendations 
state the need for Semantic Web-based approaches. On the one 
hand, the Learning Object Metadata (LOM) standard 
recommends the annotation and classification of learning 
objects, using metadata to facilitate their retrieval. On the other 
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hand, The IMS LD specification, which describes the learning 
process that takes place in learning units, is currently described 
using the ontology (Amorin, Lama, Sanchez, Riera, & Vila, 
1995). 

However, the development of ontologies has never been easy 
and getting such a degree of shared semantics in the web is a 
task that will take a few years. That is why Linked Data have 
been proposed as a way to facilitate data exchange in the Web. 
Linked Data permit using the Web to connect related data. 
According to Wikipedia, it describes a recommended best 
practice for exposing, sharing, and connecting data, information, 
and knowledge on the Semantic Web using URIs and RDF. 


complicated environments and organize knowledge so that 
a Semantic Web tool can be created in future. Therefore, 
making a fixed, immovable structure of content seems to be 
really difficult in certain contexts. That is why soft 
ontologies can be very practical in areas such as teaching. 

— Hard/strong semantic technologies support efficient 
exchange and processing of semantic data between 
programs and machines. An example would be linked data 
constructed from RDF statements. These ontologies are 
structured by means of specific languages such as RDF or 
OWL; they are usually developed through ontology editors 
such as Protege. 


2 METHODS: INFORMAL AND FORMAL 
REPRESENTATIONS OF DATA AND 
KNOWLEDGE 

This section provides more information about the nature and 
properties of ontologies that can turn out to be useful within 
learning environments; that will allow us to introduce how 
knowledge is used by the two research projects analysed in later 
sections of this paper. 

The formal notion of ontology means that ontology 
knowledge is formal; it has a specific structure with a particular, 
deterministic, and inflexible meaning. This is probably a 
required property of ontologies in certain domains and 
applications, but it limits their applicability too. Our attention in 
this paper will also be placed in dealing with the view that 
consider ontologies as a way to structure information that 
provides more powerful tools than relational models, for 
instance. 

2.1 Results and discussion: Types of Ontologies 

The Semtech (Semantic Technologies for Learning and 
Teaching) draws a distinction between hard semantic 
technologies and soft semantic technologies according to their 
structure level. An in-depth look at this topic reveals mixed 
views about soft ontologies. Some authors use the term 
lightweight (Corcho, Fernandez, & Gomez, 2003) while others 
speak about soft ontologies. In any case, it seems that both terms 
try to unite apparently irreconcilable ontological differences 
through contextual analysis. Consequently, when an attempt is 
made to represent knowledge in a structured way, disciplines or 
subjects often appear that cannot be structured in a clear 
taxonomy. This is where soft ontologies come into play. 

SemTech makes a distinction between the so-called soft and 
hard semantic technologies and between linked data and 
'traditional' metadata using ontologies like those stated in 
Tiropanis et al., (2009) (the main differences between soft and 
hard ontologies can be seen in Figure 1): 


Figure 1 . Differences between soft/light ontologies and hard/strong 
ontologies 


Soft/light ontology 


Subjective 

Specific 

Does not have a special 
computer language 
Takes a combination of 
tools and programs 


Hard/strong ontology 


— Objective 

— Common 

— Can be developed using 
OWL 

— Can be done using just 
an ontology editor (such 
as Protege) 


As shown above, it is very difficult to find an agreement on 
common vocabularies and shared conceptualizations even in 
restricted domains. Different agents use the same word to mean 
different things or use different words to mean the same thing. 
This is what Bouquet, Dona, Serfafini, and Zanobini (2002) call 
semantic heterogeneity, namely a situation in which agents do 
not understand each other because they use languages with 
heterogeneous semantics. 

A local ontology cannot be exclusively seen as a 
disadvantage, since it allows us to work with specific 
environments that can play a fundamental role in education. 
However, this perspective needs to be considered when an effort 
is being made to build and organize information. It could be a 
tremendously useful tool, but only within a given environment, 
i.e. the one for which it was developed. In fact, a local ontology 
can be either soft or hard, as it is the contextualization of its 
contents and not its degree of formalization and axiomatization 
that determines this classification. While formal ontologies 
assume that their knowledge is valid in any possible world and 
interpretation, local ontologies are supposed to contain valid 
knowledge in a particular context of use -which brings them 
closer to application ontologies. 


— Soft/light semantic technologies let people document 
certain concepts in formats that can be easily 
communicated to other people as part of learning and 
teaching processes. Examples of soft semantic annotation 
include folksonomies and topic maps. These ontologies 
refer to a flexible ontology that organizes concepts and 
ideas into a comprehensive taxonomy or classification 
(Aviles, Diaz-Kommonen, Laipainen, & Piertarila, 2003). 
Having a soft ontology does not mean less effectiveness. 
There are areas where it is difficult to establish clear 
relationships. The development of soft ontologies or 
lightweight ontologies allows us to work within these 


3 SEMANTIC WEB TECHNOLOGIES FOR 
ASSESSMENT: OELE 

The assessment of students’ knowledge or skills is a basic 
activity both in traditional education and in e-learning. This is 
usually done by giving students tests which can contain different 
types of exercises, such as open questions, closed questions, 
puzzles, matching games and so on. Each type of exercise tries 
to evaluate a different ability level of students. Bloom (1956) 
developed his taxonomy with six levels of intellectual 
behaviour: evaluation, synthesis, analysis, application, 

understanding and knowledge. Cognitive memory or fact 
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recognition represents the lowest level, whereas the highest level 
corresponds to evaluation, which requires a more abstract and 
complex problem-solving ability. The aforementioned taxonomy 
pennits to classify abstraction levels in the questions and 
exercises used to assess students' work. 

Different authors agree with the statement according to which 
the higher levels of Bloom's taxonomy can only be evaluated 
through open questions (Birenbaum, Tatsouka, & Gutvirtz, 
1992; McGrath, 2003; Mitchell, Aldridge, Williamson, & 
Broomhead, 2003; Palmer & Richardson, 2003). These 
questions are not difficult to design for teachers, although their 
manual evaluation is difficult and sometimes based on 
superficial properties of the answer, such as the presence of 
important tenns. In this case, students may easily cheat the 
assessor by writing general lines and a few senseless contents 
with the terms that the assessor is looking for. An appropriate 
evaluation process must be based on the careful reading of 
answers, looking for clarity and logic. Moreover, this task 
becomes exhausting if the evaluator has to mark a large number 
of exams. Nevertheless, the assessment of open questions 
without human participation becomes a serious issue before the 
need to evaluate a natural language text and requires the 
development of new methodologies to support such processes. 

Different techniques have traditionally been applied to the 
assessment of open questions. In particular, knowledge 
representation techniques such as semantic networks or lexical 
conceptual structures (see Devin, 1998; Olsen, 1998; 
Whittingdon & Hunt 1999, amongst others) can be found. In 
recent years, Topic Maps (Maicher & Park, 2005) have been 
widely used for the conceptualization of domains within 
educational settings. Topic maps can represent information using 
topics, associations (which represent the relationships between 
them) and occurrences. They are thus similar in many aspects to 
semantic networks and to both concept and mind maps. 
However, their knowledge is not fonnalized and requires the 
definition of the topic map ontology. 

Computer-assisted assessment systems for students can also 
be found (Friedler & Shneidennan, 2008; Falquet & Mottaz, 
2004; Alfonseca & Perez, 2004). Most of them combine natural 
language processing and statistical techniques to deal with 
students' answers. Finally, our group has already used ontologies 
to support the assessment of individuals in group work 
(Femandez-Breis et al., 2007). 

Feedback is obviously related to assessment, because it 
permits to give back to the student information which was 
provided by the latter student -and which has now been 
improved thanks to the processing carried out by the teacher. 
Feedback is not only an instrument that has to be utilized to 
improve students’ learning; it must bring an improvement of the 
whole teaching-learning process. Collecting students’ feedback 
is a central strategy to monitor teaching and learning 
quality/standards in higher education institutions. 

Considering all the above, the developers of the OeLE 
platform found the following reasons for applying ontologies to 
support assessment processes: 

Ontologies can provide a precise semantic specification of the 
domain: in this case, the knowledge which students must acquire 
through the course. 

Semantic annotations can be used to get a precise semantic 
specification of questions and answers. 

Automatic feedback processes can be developed by 
combining course ontologies and semantic annotations. 


3.1 Description of the OeLE platform 

The OeLE platform resulted from the joint work carried out by 
two research groups from the Computer Science and Education 
Faculties at the University of Murcia (Southeast Spain). This 
platform allows running assessment tests that go beyond the 
multiple-choice test traditionally used online. One of the great 
features that it offers is the possibility of taking concepts 
developed by students and using them to provide feedback about 
those students’ perfonnance in the test, indicating what things 
were done well, the errors, etc., and ultimately to provide 
feedback on the learning process as a whole. 

Ontologies in OeLE represent the knowledge that the students 
have to acquire in a particular course. Course assessment of the 
course is done by giving students tests based on open questions. 
Each open question is associated with a specific set of semantic 
annotations, which are in turn the parts of the course ontology 
that students should have acquired to answer it correctly. 
Semantic annotations are also associated with students’ answers 
in natural language, this being a semiautomatic part of the 
process where the support of natural language processing 
techniques plays an essential role. The mark will then be 
proportional to the semantic similarity between both sets of 
annotations. 

OeLE aims to support the design and improvement of 
assessments tests in e-learning-type courses. This approach is 
based on the following assumptions which reveal the need for 
Semantic Web technologies to achieve our goal: 

— Course knowledge can be modeled by means of ontologies, 
that is, using concepts, relationships and attributes. In 
particular, ontologies in this work are expressed through the 
Ontology Web Language (OWL) (Web Ontology Working 
Group, 2004), which acts as the “de facto” standard. 

— Semantic annotations can be associated with assessment 
questions. Annotations will be defined for a particular 
question at two levels: the answer expected by the teacher 
and the actual answers given by students. 

— Exams are evaluated using a methodology based on 
semantic similarity measurements. This similarity is 
calculated from the expected answer and the answer 
actually given by the student. The similarity is calculated 
with the semantic annotations extracted from both answers. 

In OeLE the exams done by students are annotated with 
respect to the course ontologies and those OeLe contain modules 
which pennit the automatic calculation of a student’s mark 
through the comparison of annotations respectively associated 
with the students’ answers and the expected answer -defined by 
the teacher. In addition to this, OeLE allows students and 
teachers to receive feedback information through the semantic 
analysis of the answers given by students in an examination. 
That semantic analysis is performed in the context of the course 
ontology. The platform permits to implement different marking 
policies and strategies by changing values in the set of 
parameters used for calculating students’ marks. 

Some OeLE screenshots are described next. The teacher uses 
the screen shown in Figure 3 to evaluate each question. This 
screen is divided into two halves. The left hand-side refers to the 
question, whereas the right hand-side refers to the answer 
provided by the student. The following information appears for a 
question: description; expected answer (optional); the semantic 
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annotations making explicit what the students are expected to 
answer; and the marks assigned to this question. Concerning the 
answer, the following information is provided: the text answer; 
the semantic annotations derived from the text answer; and the 
marks given to students. Two fields are provided for this issue: 
manual marking (done by the teacher) and automatic marking 
(calculated by the system). The final mark for the exam results 
from combining the marks obtained in each question. 

Feedback is generated in OeLE for any answer given by a 
student in an open question as described below and partially 
shown in Figure 2. The following process is executed for each 
semantic annotation of the student’s answer. The degree of 
semantic similarity between one semantic annotation of the 
student’s answer and all the annotations of the expected answer 
belonging to the same ontological category is obtained; this 
results in a table where rows are the annotations of the student’s 
answer and columns the annotations for the expected answer. 
Each cell contains the semantic similarity value. 

Figure 2. Marking the question 


Rctpucsto a Prcgunto Ablcrta Cor retdr ’ X I 



The result of this analysis is composed of two lists: 

— Knowledge not acquired (“aspectos a mejorar [aspects to be 
improved]”): this list contains the knowledge items that 
were expected to be answered in this question but were not 
provided by the student. Figure 4 shows that the student did 
not give an answer for the concepts “design bases”, “design 
phases” and “recommendations,” the relationships “design 
bases are the bases of pedagogical design” and also “design 
bases are the bases of technical design,” and, finally, the 
attribute “main aspects of design bases.” 

— Knowledge contained in the answer (“Items respondidos 
por el alumno [Items answered by the student]”): the 
marking process obtains a set of semantic annotations from 
the student’s answer. The feedback is then generated by 
showing the correctness of each ontological entity extracted 
from the student’s answer. Figure 3 only shows the concept 
Tools (“herramientas”) which was correctly answered by 
the student. Wrong items have a red cross next to them. 

The feedback generated by the platform also allows teachers 
to know which aspects have been acquired best and worst by the 
students. Figure 4 shows how OeLE displays the analysis of 
students’ exams graphically. 


Figure 3. Excerpt of the feedback generated by the student 

OeLe 

Ontology eLearning 


curso: ItEducaaon examen: DISENO DE MATERIALES 


1.- (2 puntos) tCuales son los aspectos principales a tener en cuenta en el diseiio de 
materialescon nuevas tecnologias? 


Calificacion de la pregunta 1 : 0,29 


Respuesta esperada:_ 

Ya sea con nedios tradicionales o sea con nuevas tecnologias, los » 

aspectos principales son similares. Asi, debemos tener en cuenta en 
el diseno de materiales con Nuevas Tecnologias lo siguiente: bases 
del diseno, fases del proceso, recomendaciones del diseno y 


Aspectos a mejorar 


Aspectos No Adquiridos 



3.2 Using OeLE in real settings 

OeLe was used during an e-Learning course at the University of 
Murcia. The course took place in the second semester of 
2008/2009 with a group of 25 students. This subject has a 
practical orientation and has as its main goal to train the students 
in the aspects of content design, production and evaluation 
within the framework of didactic processes. All the work is 
realized in the virtual e-Learning platform at the University of 
Murcia: SUMA (http://suma.um.es/). 

Students’ academic performance was assessed with an e- 
portfolio and other different activities throughout the 9 units in 
the program and also taking into account the participation of 
students in several communication scenarios (videoconferences 
and collaborative work activities). The students also had to take 
a written exam in a specific assessment environment. OeLE was 
the assessment environment used in this course, and our research 
was focused on this part of the course. The rest of variables 
defining the course were taken into account when designing, 
analyzing and evaluating the experience, but the attention was 
mainly focused on testing the educational usefulness of this 
innovative assessment and feedback system. 

The experiment was executed as follows. Students took an 
exam using the OeLE platform, after which exams were assessed 
by OeLE, thus generating the feedback infonnation for each 
student as well as for the teacher. The exams were marked by 
one of the course teachers too. At that stage, students were given 
access to reinforcement contents for one week; in other words, 
students had the chance to revise the contents associated with the 
course knowledge items that they had not answered correctly in 
the exam. Then, those students took a second exam which was 
marked exactly as the first one. Finally, the students were asked 
to fill in a questionnaire about the experience, which permitted 
to analyze the experience as a whole. 

The data analysis revealed that the marks in the second exam 
where higher than in the first one. This is an expected result 
because considering that feedback was provided to students and 
the second exam took place only a few days later. Our analysis 
shows that feedback was one of the most highly assessed aspects 
in this experience. These results are in line with those found in 
previous works (e.g. Gordijn & Nijhof, 2002; Wang & Wu, 
2008). Most students considered it positive, and a positive link 
was identified between the idea that feedback is useful in online 
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assessment environments and the usefulness assigned to the 
feedback received in the experience. 

The feedback generated by the platfonn allowed teachers to 
know which aspects had been acquired best and worst by 
students. This seemed interesting to teachers because each 
knowledge item has a relative importance associated to each 
annotation in the questions. In this experience, the best acquired 
concepts were those with a higher relative value. 

3.3 Methodological issues 

This experience allowed us to define the methodology for using 
this kind of semantic web approach to support assessment 
processes. 

The recommendations of various authors (Noy & 
McGuinness, 2001; Devedzic, 2007; Fernandez et al., 2007; 
Horridge, 2009) should be taken into account when working 
with ontologies. 

The program Protege represents the best option to create the 
ontology. The ontology has to model the course knowledge in 
terms of teaching and asking, and paying attention to the way in 
which the contents are related. It is advisable to establish 
connections between concepts. Answering the following 
questions posed by Noy and McGuinness (2001) can help when 
it comes to create the ontology: 

— What is the ontology domain? 

— How will the ontology be used? 

— What types of questions in the ontology should provide 
answers? 

— Who will use and maintain the ontology? 

In the course of the OeLE experience, different meetings were 
held with the subject lecturers in order to define what aspects 
they took into account to evaluate students' answers. Aspects 
such as clarity, simplicity and originality in the response (to 
avoid copy-paste) and provision of examples were the most 
outstanding. These aspects were also included in the ontology. 


Figure 4. Example of work with Protege 



The next step is to list important tenns in the ontology. For 
this purpose, it is useful to write a list of all the terms about 
which statements are going to be made or explanations are going 
to be given to the user. There is consequently a need to ask 
ourselves what the important concepts are. 


The way to create the ontology is through the use of classes, 
sub-classes and the relationships between them. It is worth 
highlighting that a hierarchy will be established. The ontology 
refers to the exam questions, so lecturers might be tempted to 
organize the ontology taking the questions as their essential 
reference. Our advice is to create the ontology based on 
contents, though not on those corresponding to entire course; the 
prototype of answers created for our review should be the 
reference. 

4 DEVELOPMENT OF EDUCATIONAL 
SEMANTIC WEB APPLICATIONS: 

ENSEMBLE 

Exhibit is a tool inside the SIMILE project (Semantic 
Interoperability of Metadata and Information in unLike 
Environments). This project aims to enhance inter-operability 
between digital assets, schemata/vocabularies/ontologies, 
metadata, and services. They seek to provide end-user services 
by drawing upon the assets, schemata/vocabularies/ontologies, 
and metadata held in such stores 

(http://simile.mit.edU/wiki/SIMILE:About). Exhibit is a data 
linking/aggregation tool but it gives you some of the affordances 
of the semantic web without the need for ontologies. Exhibit 
allows you to easily create web pages with advanced text search 
and filtering functionalities with interactive maps, timelines, and 
other visualizations. 

Hyun, Kragen, and Miller (2008) specify that the Semantic 
Web has been dismissed by many specialists because it can 
show an unrealistic view, since it was launched as a resource in 
which automated software agents can collect structured data 
carrying out complex tasks on behalf of users. 

4.1 The ENSEMBLE project 

Our next focus of interest is the different educational contexts 
and the role that ontologies can play in them. The Semantic 
Technologies for the improvement of problem-based learning 
project (ENSEMBLE) is a good example of how to work 
differently with the information and develop Semantic Web 
applications. This project directed by Patrick Carmichael stands 
out as one of the major international projects that studies the 
application of the Semantic Web in Education and does research 
into case-based learning and the Semantic Web. 

What mainly differentiates it from other research projects on 
the Semantic Web is its educational perspective. It tries to 
analyze the implications associated with the use of Semantic 
Web technologies in education, while many research groups 
focus exclusively on technical issues. The project works with 
teachers, undergraduates and graduate students and tries to 
identify the aspects in which the Semantic Web can improve 
their learning as well as the role that Semantic Web technologies 
and techniques can play in supporting such learning. 

Martinez, Morris, Tracy, and Carmichael (2011) explain how 
they developed several applications for different areas (plant 
sciences, archaeology...) through the exploration cases. The 
project has been developed through a wide range of semantic 
applications especially focused on teaching. 

The dance project stands out among their many projects. Due 
to its context and contents, this project required the extension of 
frameworks for the tools with which they worked. They created 
a web environment containing interrelated information where 
students can interact with applications and information. It is a 
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good example of how important resources can support 
description using formal vocabularies that provide a means of 
communication across settings and contexts (Martinez et al„ 
2011). Processes such as information interaction and connection 
can illustrate a way of applying the Semantic Web to education 
without ontologies. The particular benefits from semantic web 
technologies that they describe on the website are: 

— Consistent metadata allows content to be stored and 
retrieved from a digital repository through web interfaces. 

— Faceted browsing allows searching for collections, and as 
in the first demonstration below, exploration of the 
relationship between formal ontologies and informal, user¬ 
generated vocabularies. 

— The semantic “mark-up” of video content, linking it to parts 
of performances, other resources and terms. 

— Annotation of contents (including video) makes it possible 
for students to assemble their own collections and supports 
the “case building” described above. 

Figure 5. Dance photo viewer 

( http://ensemble.limu.ac.uk/proiects/tests/TestDance/index.html l 


Dance Photo Viewer 

1 08 iwnanMi Performance 



Another example is MOAM (Maritime Operations and 
Management), a course on maritime operations and 
management. This is a Master’s course imparted at the City 
University London. In this course, students work on real cases 
with different practical assignments that formulate a problem to 
which they need to give a practical solution. The course includes 
several learning scenarios, for example, a task may be to design 
a ferry service from the islands to the coast of Cornwall (Stcholl, 
Tracy, & Carmichael, 2009). Students must play the role of 
experts in different domains (e.g. economists, security engineers, 
technical designers), presenting and defending their work before 
a panel of experts. This way of using cases requires an approach 
where a reflection is made on technological support for semantic 
tools. 

4.2 Methodological issues 

Some supporters of a traditional Semantic Web perspective 
might ask where the OWL ontologies are and how we can 
include them in these open models. Hyun, Krag, and Miller 
(2008) give us the answer indicating that they have chosen a 
different syntax model, which is not based on RDF/XML, and 
also that they accept the limitations derived from not working 
with such languages because they can take advantage of the 
Exhibit work method, thanks to which faster, more practical and 
user-friendly applications may be developed. 

For them, it is counterproductive to generate automated forms 
from the OWL code. In their view, the critical first step towards 
the generalization of the Semantic Web is to generate large 
amounts of data, and Exhibit is a tool that creates incentives for 
authors to make this happen. The rationalization of the data can 
come later. 


The ENSEMBLE project describes how to create a semantic 
web application in five easy steps on its website 
(http://ensemble.ljmu.ac.uk/): 

— Step 1: The initial spreadsheets. Work should start with a 
handful of basic data tables, each containing rows that 
detail different types of data (philosophers, sessions, 
readings...) linked to provide semantic associations. 

— Step 2: The ‘people’ spreadsheet run through Babel. 
Working with the data is only possible after changing them 
from spreadsheets into a record format that your web 
browser can understand. The SIMILE project (home to 
Exhibit) hosts an online service to do exactly that: Babel. 
The service can also create very crude mock-up Exhibits 
from the spreadsheets that are fed into it. This is a useful 
tool for visually checking the correctness of our data before 
beginning to work with a full Exhibit version ourselves. 

— Step 3: The ‘themes’ page as a skeleton Exhibit. Once the 
data have been obtained from Babel as JSON (JavaScript 
Object Notation, a data file format often used by web 
applications), it is possible to start building our Semantic 
Web site. The first step is to throw up a view and a few 
facets to check that our data are fit for our purposes. 

— Step 4: The ‘themes’ page with proper lenses. After 
checking that the skeleton is appropriate, it can be further 
developed using lots of Exhibit features, like HTML lenses 
to control how each record is shown. 

— Step 5: The final site. The great thing about semantic data is 
that it is possible to view them and manipulated in many 
different ways as long as they have plenty of solid 
connections to link different records. The same data sets 
were used on the final site over and over again to drive 
several learning resources, all of them created using the 
same Exhibit tools. 

The really interesting part of the aforementioned experience is 
the effort made to work in trying to integrate ontologies within a 
context related to a specific learning. As could be seen at the 
beginning of this section, soft ontologies try to put together 
apparently irreconcilable ontological differences through a 
contextual analysis. This means that when an attempt is made to 
represent knowledge in a structured way, disciplines or domains 
often appear that cannot be structured by defining a clear 
ontology. Taxonomies related to structured knowledge 
combined with the utilization of other tools can help solve this 
problem. 

One possibly appropriate way to represent information in this 
situation is to develop the ontology on the basis of Bloom’s 
taxonomy. As shown above, Bloom’s taxonomy represents a 
way to classify instructional activities or questions as they 
progress in difficulty. This taxonomy provides a useful structure 
where test questions can be categorized in the assessment of 
student learning. 
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5 DISCUSSION AND CONCLUSIONS 

In practice, ontologies have become widely used because they 
are reusable and shareable (Brewster & O’Hara, 2007). The 
opportunity to work in different projects leaves us in a privileged 
position when analyzing both perspectives about the possibilities 
to work with ontologies in educational environments. The 
educational advantages and opportunities offered by both OeLE 
and MOAM experiences are explained below: 

I. OeLE: 

a) OeLE application is an innovative assessment system 
which has as its biggest advantage that it provides 
students with mechanisms to obtain infonnation about 
their weaknesses. It also provides teachers with 
mechanisms to ensure that their students meet their 
individual and collective perfonnance goals as well as 
to react effectively with the ultimate aim of improving 
student learning. 

b) Building ontologies is actually a time-consuming task 
cost; however, creating the ontology immediately 
brought the advantage of obtaining an explicit 
categorization of the elements and relations involved in 
the knowledge model, so that the model could be 
edited, managed and reused. Moreover, the semantic 
organization of feedback by learning objects implies 
the possibility of a future interrelation of several 
computer systems and is a more effective to manage 
both students’ and teachers’ contents. 

c) Once the ontology has been developed, it can be reused 
in different editions of the course and the base of 
semantically annotated questions can be shared with 
other teachers and used in different course editions. 

II. MOAM: 

a) MOAM offers the possibility of working with specific 
learning environments and making useful tools, thanks 
to which it is possible to have broader domain 
representations. Based on standardized elements (such 
as Bloom’s Taxonomy) the MOAM experience allows 
us to create future work patterns to understand how the 
different elements interact in an educational context. 

b) The creation of working patterns allows us to know 
how tasks are perfonned in daily practice seeking to 
achieve specifically fixed learning goals. It is also 
possible to watch the patterns that respond to higher 
goals. 

The experience in OeLE presented here means that there are 
situations in which the knowledge of certain elements could be 
clearly structured, while an experience like MOAM or dance has 
as its aim to include other contextual elements (taking over the 
idea of soft-ontology). Therefore, different elements can be 
distinguished in each situation. Although an attempt was made 
by us to include contextual elements in the OeLE experience, 
content representation is completely structured from the "answer 
model" that the teacher provides. 

The following table shows the aspects developed with regard 
to different items: 


Table 1. Differences in different structured knowledge 



Well-defined 

knowledge 

Heterogeneity-knowledge 

DOMAIN 

Knowledge, e- 
leaming 

Methodology, skills, 
outcomes 


Already set or 


LEARNING 

OUTCOMES 

predefined from the 
start, learning 
outcomes at the same 
level, same 

Emergent, contested and at 
different levels and using 
different vocabulary 


vocabulary 



Representation, 

Articulation, 

ONTOLOGY 

Assessment, 

Representation, 


Feedback 

Translation 


— Domain: The knowledge that is important in an e-learning 
course must be organized in OeLE. The process is 
methodical and tries to prioritize some concepts over 
others. At the same time, it is important to find the most 
representative word of an idea and find synonyms that can 
help in future for experiences with well-defined knowledge. 
Instead, approaches like MOAM or dance focus on trying 
to find how to define the skills needed to understand the 
operation of a course, i.e. to know what content responds to 
the aim sought and what skill is needed to develop it. 

— Learning outcomes: The learning outcomes in a well- 
defined knowledge experience must be fully standardized; 
they need to share a common vocabulary and provide 
alternatives that computers can understand. At first, 
learning goals are at the same level and it is up to the 
teacher to choose the most important one. The learning 
goals in other contexts are recent elements operating at 
different levels and there is a permanent attempt to improve 
and learn. This strategy requires deepening into the subject, 
so it is basic to become familiar with the methodologies, the 
objectives and the assessment system. OeLE also takes into 
account these aspects but, for MOAM, this study of 
learning goals is the pillar of the future application. OeLE 
ontologies may be used in different subjects while MOAM 
is more specialized and localized. 

— Ontology: An OeLE-based ontology is created through 
tools like Protege and exported in OWL formats which 
facilitate the execution of assessment and feedback 
processes as well as the association with Learning Objects. 
With a knowledge heterogeneity experience where we 
should take into account learning objects or other elements, 
the ontology stems from a set of structured information 
which is translated into a computer language so that we can 
relate every element specific to a case. This representation 
is more complicated from a technical point of view but it 
offers a more realistic view for the teaching of science. 

— Technologies: OeLE bases its goal on creating a system for 
evaluating courses in e-leaming environments, while 
MOAM aims to deepen into the benefits of an educational 
reality and include the Semantic Web as a part of teaching. 

To the best of our knowledge, none of the existing approaches 
had so far succeeded in supporting the assessment of open 
questions using a complete semantic infrastructure before OeLE. 
OeLE provides the academic community with a new tool- 
assisted method for the assessment based on open questions. 
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This approach shows the value of Semantic Web technologies 
for building e-Leaming solutions. 

Hence, it is worth reflecting on the benefits can be drawn from 
both experiences. It cannot be said that these two experiences 
are completely at odds, since a number of possibilities can be 
explored in joint future projects! 

— Creation of more realistic e-learning assessment systems 
adapted to the context 

— The combination of OeLE evaluation as a platform to 
search for learning goals and activities for the teaching 
practice. 

— The creation of a complementary tool to OeLE that allows 
the combination of Learning Objectives with learning goals 
and their implementation in networked assessment 
environments. 

There is not a single exclusive way to work with ontologies 
nowadays. The first step should be to analyze the type of content 
and context in which work is being carried out. After doing that, 
it is possible to work with different tools ranging from more 
structured and closed ontological models (with programs such as 
Protege) to more open models where other tools are utilized to 
structure information. 

There is no doubt whatsoever that something is happening at 
the Semantic Web. Several lines in which work could be 
undertaken combining both perspectives are open for the future. 
One of the most outstanding ones is the development of an 
OeLE tool including elements from soft ontologies such as 
MOAM works, which offer a more realistic view of the subject. 

It should equally be remembered that the European Higher 
Education Area, as well as the reform of university degrees in 
progress, also seeks to change teaching methodologies by 
orienting them towards learner-based strategies. There is 
additionally a full awareness of the relevance that e-Learning 
has in European projects. That is why none of us can afford to 
miss out on this chance to study these new educational 
opportunities from a multidisciplinary perspective. 

To conclude, it is worth highlighting that the development of 
this tool shows how a more complete online assessment 
environment can be created through the OeLE application and a 
networked feedback system supported on Learning Objects, thus 
generating an innovative working system for e-Learning courses 
or subjects. 
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